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ABSTRACT 

This paper presents mu lti-module focal plane processin g sensor 
architecture which provides high resolution (up to 512 x 512 
pixels) muJtiscale real time analog e dge extraction for robot vision. 
The hexagonal CMOS sensor uses a multiport addressing 
architecture of the pixel amy in order to apply external multiscale 
analog spatial convolution followed by edge detection. The sensor 
architecture and its peripheral analog filtering modules are 
described and relevant results obtained from an actual 256x256 
prototype are presented. This analog satellite processing approach 
may be extended to various types of computational sensors 
including 3D range finder, motion sensors or tactile perception 
devices. This multi-module approach allows the implementation 
of high resolution sensors with a very powerful computational 
capability tailored to computer vision applications. 

1. INTRODUCTION 

Real-time computer vision is a challenging research area for which 
several custom integrated circuits have been developed in order to 
achieve higher performance for pattern recognition tasks. It is 
well-known that major computational resources are needed for 
low-level image processing like enhancement or edge detection 
and especially for multiscale edge analysis. Although DSP 
processing and architecture were integrated to frame grabbers in 
order to support low-level image processing, the resulting 
acquisition system must still deal with a large amount of data with 
little capacity of flexible decision making on what is relevant or 
not in the scene. In addition, an external video camera must be 
added to the acquisition system and imposes several functional 
constraints like frame duration and fixed scanning path. 
Computational sensing is an emerging research area which 
combines photo- transduction and signal processing at the sensor 
level. In a computer vision perspective, this means that analog 
illuminance signals from a 2D sensor may be used by an analog 
computing module in order to_ex tract rek van; characteristics from 
the scene. A major consequence of this approach'ls" a significant 
reduction" of data transfer between the sensing unit and the 
recognition processes. 

VLSI technology provides opportunities for the design of a 
variety of image sensors. An intuitive design strategy refers to 
retina -like architectures with complex photo-sensitive elements 
[7] and emphasis on the communication between neighbors [3] [4], 
but it usually defines sensors with poor spatial resolution. The 
parallel access of row data at one end of the sensor [2] or an on- 
chip digital sequential processor [1] have been previously studied. 
Although these approaches, based on digital theory, are more 
compact designs than the conventional frame grabber approach, 
they still offer similar computational power. 

A common goal to computational sensing approaches is to 


integrate photosensitive elements and analog processing on 
CMOS technology [8] [111 due to its good yield and spatial 
regularity. Good 2D image sensor must provide a minimum spatial 
resolution (typically 200 to 400 pixels per line) when computer 
vision is targeted as the application. This implies a basic trade-off 
between pixel complexity (size) and the spatial resolution that is 
allowed by a given technology for maximum die size. In this paper, 
an hexagonal CMOS sensor is described with its multi-port access 
architecture. This allows parallel analog extraction of illuminance 
data which may be processed externally by a satellite analog 
processing module. With this architecture, it is possible to perform 
multiresolution edge extraction using several external integrated 
filters in parallel without any negative effect on the overall 
resolution of the sensor. This solution requires different IC 
technologies on separate dies which makes it a typical candidate 
for a multi-module chip assembly. 

The concept of parallel analog extraction using an nexagonal 
Multi-port Access photo- Receptor (MAR) sensor is presented in 
Section 2, including the pixel architecture and sensor operating 
mode. The parallel analog spatial filtering capability of the MAR 
system is presented in Section 3 with a typical application for 
multiresolution edge extraction using custom resistor network IC. 
The paper concludes in Section 4 with a presentation of parameters 
obtained from the analog computing module and typical images 
from a 256 x 256 prototype of the MAR sensor. 

2. THE MULTI-PORT ACCESS PHOTO-RECEPTOR 

A multi-port addressing strategy was chosen for the sensor in order 
to access, in parallel, a large number of pixels within the sensor but 
keeping the pixel size as small as possible. The analog processing 
is transferred to an external custom IC. An hexagonal grid has been 
selected for the MAR sensor for two main reasons. Firstly, it is 
relatively easy to extract analog data by a set of parallel busses. 
The second reason is related to the distance between a given Pixel 
Of Interest (POT) and its nearest neighbors which is the same along 
any of the six main directions of the hexagonal tessellation, a 
condition which considerably facilitates the implementation of 
circularly symmetric kernels. Figure 2 shows the schematic of 
each pixel which includes six transistors and one photo-diode. The 
single current / s , generated by the integration of photo-current 
is retrieved through a set of three N- transistors {f4 r , M st and 
Mjq) with individual selection lines. This is similar to a multi-port 
memory access with individual data busses (JDy* ^Xl D X2 
respectively). Each set of selection busses (X, X t and X 2 ) » 
activated by an individual bidirectional shift register. Each signal 
is routed out via one of these data busses, according to the status 
of the selection lines. The dark area on top of the convolution 
kernel on Figure 1 represents the extracted analog data flow until 
it reaches the parallel analog multiplexor. Pointer T, which tracks 
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Figure 1 General architecture of the MAR sensor and its 
associate external muliire solution analog filtering unit, A 
Typical addressing of a pixel of interest is shown on the 
sensor. The white star represents the convolution kernel 
which extract illuminance information for 9 J pixels. 


the location of the POI, controls a set of transmission gates that 
puts each analog signal on the proper output channel. 

This topology allows for simultaneous access to the 
illuminance data of the selected pixel (intersection of the three 
active selection lines) together with the illuminance of all 
neighbors located on the three axes of symmetry of the sensor 
array (corners of the concentric hexagon). It yields a natural 
compatibility with circularly symmetric operators since all pixels 
located at the comer of a given hexagon have identical radial 
distance from the POL A particularity of the MAR sensor is that 
the POI may be moved 'along any of the axes of the underlying 
hexagonal structure which allows the sensor controller a variety of 
scanning strategies [10]. 

Since each pixel may be accessed several times during a scan 
period, the read-out must be non-destructive as shown in Figure 2. 
The circuit of Figure 2 is based on a photo-current I E generated 
by a photo-diode which drives the gate capacitance of transistor 
Mj over an adjustable inte gration interval. The integration process 
is initialled iBeut ~ 1 Grab = 1) by placing a positive voltage 
(V Reset * v Sn) through Mi and M j on the gate of My. Transistor M 3 
is disabled to stop the integration process during the scanning of 
the sensor in order to allow uniform integration time throughout 
the sensor. 

3. MULT1KESOLUT10N CONVOLUTION MODULE 

The most interesting property of the proposed architecture is that 
it allows parallel analog filtering for edge detection it multiple 
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Figure 2 The schematic view of the MAR pixel. The right- 
hand part refers to the multi-port addressing architecture 
while the left-hand side shows the non-destructive buffering 
of the analog illuminance data. 



A S,= ^(2nr) 

Figure 3 The information from each set of six pixels which 
are located at the same radial distance from the POI is 
routed out from the sensor on individual data buses in order 
to apply circularly symmetric filters. 

scales. As previously discussed, the hexagonal tessellation is 
naturally compatible with circularly symmetrical operators. The 
Laplacian Of Gaussian (LOG) operator has been identified by 
Marr et al. [5] [6] as a most natural isotropic edge detector. 
Unfortunately, their computational cost on sequential computers is 
quite high. On the MAR architecture, the computation of 10 or 20 
simultaneous 61tered images is straightforward. It is the most 
relevant property of the MAR architecture, especially when scale- 
space integration is planned as an edge reinforcement procedure. 

Figure 3 gives an overall view of the star-shaped convolution 
kernel of the MAR sensor which has a maximum radius of 16 
pixels from the POI (/ 0 ). Each extracted current /, is put on an 
individual analog output channel, where r defines the radial 
distance of the pixel from the POI. Since each analog signal is 
retrieved via an individual bus, it is not desirable to access all the 
91 pixels as individual sensor outputs. Rather, each set of six pixels 
with equal radius are grouped and dedicated for circularly 
symmetrica] operators. The discrete spatial convolution of a given 
image / with an operator H is expressed for the general case as: 


X Y 

Wl**y] =1 L Nli,j\Ilx-i.y-j]AS 


(1) 


with A 5, the effective area of the sampled pixel (usually a 
constant and equal to 1). For the particular case of the MAR sensor, 
where a sub-sampling of the image is made by the convolution 
kernel, we need to compute the effective area of each selected 
pixel before applying any weight H [ij] . This area is shown in 
Figure 3 (dark area surrounding a pixel) and has a value of one 
sixth of the effective ring ( A 5, = nr/3 ). The effective size of 
the center pixel / 0 is the area of a small disk with radius of J/2 
( A S 0 = n /4 ). Equation (1) may be corrected for the particular 
case of the MAR sensor with a circularly symmetric operator 
FJr]: 
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IS 


5 ^ct0]/ 0 + I rF 0 [r]l r 
r - I 


(2) 


In equation (2), the value of FJr) corresponds to the sampled 
value of the continuous function FJr) for r ranging from 0 to 15 
pixels. For LOG filters, F is a function of parameters o and r and 
is given by: 


F.l') - * CAr) = 


2 no 


a V o ) 


(3) 
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/7fure 4 Parallel analog filtering module. Each output of 
the sensor is converted to a voltage signal and processed by 
a set of resistive networks which implement N different 
filters. 

One of the most important property of FJr) is its 6rst moment 
(mean) zero. This can be written as: 


IS 


£r- 0 [r]p[r] = 0 
r-0 


(4) 


where p(r] is the probability density function of FJr} and is 
denned as p[r]= J/9J for r=0 and p{r]=6J91 for r=l,..J5. In order 
to keep this property, the sampled function FJr] is thus modified 
with a correcting function CJr] after the sampling process so its 
first moment remains lero. 

For multiresolution LOG filtering, we synthesize several filters 
computed by (3) for different values of o. As shown in Figure 4, 
each filter is implemented by a simple resistive network connected 
as an analog adder. The resistor values R a {r] are chosen such 
that their value is proportional to \/{rF 0 [r]) . The left-hand 
part of Figure 4 holds the signal conditioning module as a current- 
voltage converter. Each filter output V c is routed to a zero- 
crossing detector which transforms the corresponding analog 
output into a two-bit digital signal: (i) the sign of the analog signal 
ax j (ii) the thresholded value of its amplitude. This set of binary 
values (2N bits) are stored in an image memory and is later used 
by the digital controller. Details on the implementation of the zero- 
crossing and edge tracking algorithms are available in[10]. 

3.1. The LOG Filter as a Resistive Network 

In equation (2) the F a [r] function yields negative values. 
This is undesirable since resistor values must be positive. Hence, 
we compute a different function K a [r] = y a F a [r] + 5. m ° 
which shifts and rescales F a [r] to more convenient (positive) 
values. The operational amplifier (see Figure 4) is used to remove 
the added offset Km 0 after the convolution operation has been 
performed on the illuminance signal. In order to compute the scale 
factor y and the offset Km 0 , we must consider the layout level of 
the NTE CMOS3DLM process. 

The 272 resistors of the filter chip were buill using the same 
custom template (see Figure 5). The template has a ratio close to 
16:1. This enables the making of almost squared macro-cells, 
optimizing the use of the silicon area. 

The lowest value for any resistor can be determined by observing 
that one output of the I-V converter has to drive 16 parallel 


Adjustable (up to 54 YSl) part 


Fixed 100KG 







— tT77 



I P-well S device well 0 metal 0 P+ diffusion 
Figure 5 The template resistor. The P-well has a high 
resistivity of 4KQ. per square and the device well has a low 
resistivity of300Q. per square. 

resistors. Since the minimum load each output of the converter can 
drive is about 5KC1 , the minimum value for any resistor is SOKQ . 
We chose R ■ = 100KO to maintain a security margin. The 
maximum value for a resistor is determined by geometrical 
considerations to optimize the silicon area as explained above. The 
maximum value is R max = 154KQ. We can now compute Km 9 
and v . Let 


o *c °mtn 


a mtn 


, and 


°m<ur 


Km a +yF a = K - 

o 'o c maJ °max H. 


(5) 


(6) 


'mm 


where F a = (/% [r] ) and F = Min {F [r]) . 
The solution^ equations (5) and (6) is s&aightforward. The 
resistors values of the filters are: 


(7) 


The R values are used along with the operational amplifier to 
remove me offset. The equation of the circuit is: 


15 


to R ° 1 r] RF °t> r-Q 


15 1 1 


(8) 


where p(r] is the density function of the variables R and R a [rJ. For 
the iso-Uluminance case, that is when all the V/rA are equal, the 
property staled in equation (4) implies that V o =0 volt. We can then 
solve (8) to obtain 


R h = 

DO 


U-0 


* 0 M 


(9) 


All the resistor values are set for every filter. 
3.2. The Zero-Crossing Detector 

The analog outputs V a of the filters are routed to a zero- 
crossing detector. The analog signal is transformed into a 2-bit 
digital value. The shape of V c is shown on Figure 6(a). To detect 
the sign of V c , a simple CMOS inverter with a switching level of 
0 volt is required. To detect the thresholded value of the amplitude 
(tvao), two programmable switching level inverters, one inverter 
and one nand gate are required as shown on Figure 6(b). This is 
done by spUting and lowering the ground levels of the inverters. 
The actual values are also shown. The binary values are stored in 
an image memory. 
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Figure 6 (a) Cross-section of a convolved image, (b) The zero- 
crossing detector. The switching level of the inverters are OV, 
-IV and +lVfor VI, U2 and US. 


4. RESULTS 

Preliminary experiments of the zero-crossing detector show that its 
maxima] frequency of operation is around 40MHz. The resistors of 
the networks were measured experimentally and equations (7) 
back to (4) were recomputed with measured values. Table 1 shows 
the normalized moments of measured F a [rJ compared with 
theoretical values. This is a qualification of the rejection of 
uniform ilium inancc. A 25 6x256 pixels version of the MAR sensor 


Table 1 Normalized moments of some measured FJr}. 


Filter (o) 

Mean value 

Variance values 

Theory 

Measured 

Theory 1 Measured 

0(0.5) 

0 

-M1E-03 

0.192 

0.198 

1 (0,7) 

0 

6.06E-03 

12.4E-03 

13.0E-O3 

5(13) 

\cr» 

1.36E-Q3 

439E-06 

469E-06 

9 (2.4) 

icr 19 

269E-06 

17.0E-06 

17.0E-O6 

15 (6.9) 

1a 20 

40.2E-06 

142E-09 

I25E-09 


is currently installed in a custom camera case. This version 
implements sixteen different filters. Several image acquisitions 
have been performed on different scenes as a proof-of-concept. 
Typical results are shown in Figure 7 along with the resulting edge 
data. The upper left part is the illuminance image in an hexagonal 
sampling format and the upper right part is the resulting edge map 
following scale-space integration [10] of the sixteen primary 
multiresolution edge images. Two of the sixteen primary edge 
maps are shown on the bottom for two different spatial resolutions. 



Figure 7 Results from a typical scene for a 256 x 256 
hexagonal MAR sensor. 


(Note that black dots in the illuminance image is caused by noise 
on power lines due to the temporary installation of the Analog to 
Digital convener and are not found in edges maps). Small zoom 
boxes are used in order to show the hexagonal tessellation of the 
illuminance image an for the orientation of primary edge segments 
which are oriented in one of the third main direction of the 
hexagonal structure (at 60 degrees). The high pass filter (o=0.8) 
extracts accurate but noisy edges, while the low pass one (o=2.4) 
finds only the smooth variations of illuminance. The integration of 
these informations is made by detecting relevant edges with the 
low pass filters and localizing them with high accuracy using high 
pass filters oupouts. All of theses edge maps are generated 
simultaneously by the analog computing module in a single frame 
period. 

5. CONCLUSION 

An hexagonal multi-module image sensor has been presented and 
is designed to link acquisition with tightly integrated satellite 
processing. The feasibility of a sensor with a resolution up to 500 
x 500 pixels is an important consequence of- this concept. The' 
analog processing architecture was also detailed. A digital 
controller and several co- processing units are under development. 
This entire system will define a smart image acquisition device 
which could be used for general computer vision application an(jL 
particularly forj^rtern recognition in robot vision. 
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